Policy iteration algorithm based on experience replay to solve H∞ control problem of partially unknown nonlinear systems

— In this paper, an online adaptive optimal control algorithm based on policy iteration (PI) is developed to solve the H∞ control problem of partially unknown nonlinear continuous-time (CT) systems. The convergence of existing PI algorithms for solving the H∞ control is guaranteed under the restrictive persistency of excitation (PE) condition. By using the idea of experience replay this condition is relaxed here to a simplified rank condition which is easy to verify online. This is achieved by using previously stored data concurrently with current data for updating the critic NN weights. The proposed algorithm is implemented on actor-critic-disturbance neural network (NN) structure, where all NNs are tuned at the same time to obtain the solution of the Hamilton-Jacobi-Isaacs (HJI) equation, without requiring the information on the internal system dynamics. The stability of the closed-loop system is guaranteed and the convergence to the optimal solution is obtained. Simulation results show the effectiveness of the proposed method.